Section: New Results

Computable approximations for continuous-time Markov decision processes on Borel spaces based on empirical measures

The following result has been obtained by J. Anselmi (Inria CQFD), F. Dufour (Inria CQFD) and T. Prieto-Rumeau.

We propose an approach for approximating the value function and computing an ε-optimal policy of a continuous-time Markov decision processes with Borel state and action spaces, with possibly unbounded cost and transition rates, under the total expected discounted cost optimality criterion. Under the assumptions that the controlled process satisfies a Lyapunov type condition and the transition rate has a density function with respect to a reference measure, together with piecewise Lipschitz continuity of the elements of the control model, one can approximate the original controlled process by a sequence of models that are computationally solvable. Convergence of the approximations takes place at an exponential rate in probability.